Image processing apparatus, method and program, and storage medium

ABSTRACT

In an image obtained by reading a document or the like, when a round portion is present at a corner of a frame for describing a character, extraction of a region where a character image is described is performed by removing pixels constituting a part of rules or lines constituting the frame. Such pixels are removed from an extracted image region by identifying whether or not a desired pixel group contacting at least one of row-direction rules and column-direction rules constitutes part of the rules, by performing labeling for the pixel group.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus, methodand program for segmenting a character image described in a frame or atable in a document image, and a storage medium storing such a program.

2. Description of the Related Art

When reading characters described in a document, such as an applicationform or the like, in many cases, a document formed with a dropout coloris used, and characters described with a color different from thedropout color are read and recognized. Since dropout-color documents areprinted and distributed using a color adapted to conditions of a lightsource of a reading apparatus, the documents are expensive andinconvenient.

Although non-dropout-color documents are less expensive and convenient,it is impossible to discriminate between characters and closing lines.Accordingly, the following approaches have been considered as methodsfor recognizing a non-dropout-color document.

In a conventional method, the positions and the sizes of frames fordescribing characters within a document are stored in advance as adocument definition, and described characters are extracted by readingdifferences between an input document to be read and the documentdefinition by superposing them after correcting position deviationbetween the input document and the document definition. In this method,however, since it is necessary to store the document definition inadvance, an input document other than the document definition cannot bedealt with. Accordingly, techniques have been considered in which, evenif a document definition is not stored in advance, a frame fordescribing a character and a character within the frame are recognized.

For example, according to Japanese Patent Application Laid-Open (Kokai)No. 07-282191 (1995), a black run continuous for at least apredetermined length in the vertical and horizontal directions isextracted as a set of rules (i.e., lines). In some cases, a verticalrule and a horizontal rule are disconnected. In such a case, thevertical rule and the horizontal rule are virtually extended, and if thelength of an extended portion is equal to or less than a threshold, thevertical line and the horizontal line are assumed to be connected, andthe black run is recognized as a rectangular frame. However, since theframe is handled to be a straight rectangle, if a document having around portion at a corner of a frame is recognized, the round cornerportion is segmented as a character when extracting a character withinthe frame and is used in character recognition processing, and erroneouscharacter recognition will sometimes occur.

In Japanese Patent Application Laid-Open (Kokai) No. 07-14000 (1995), aplurality of patterns of corners of a frame are stored in advance. Blackpixels continuous in vertical and horizontal directions are checked, andblack pixels continuous in the vertical and horizontal directions,having a length equal to or more than a predetermined value aredetermined to provide rules. A corner-portion pattern that is stored inadvance is matched with a intersection point of a pair of a verticalrule and a horizontal rule whose end points are close to each other. Ifsimilarity is large, the corner portion is determined to correspond tothat pattern. In this approach, it is necessary to store a large numberof corner-portion patterns in advance, and therefore to provide a largememory capacity. In addition, a corner portion is not recognized if ithas a pattern other than stored patterns.

In Japanese Patent Application Laid-Open (Kokai) No. 2000-235619(2,000), a round corner portion is determined by identifying apixel-density distribution in an oblique direction between end points ofextracted rules, or by performing a matching operation by generating apattern connecting end points of rules using an n-degree function. Inthe former approach, however, when a change in the pixel-densitydistribution is small, for example, when a thin portion is present in acorner portion, the corner portion is sometimes not recognized. In thelatter approach, although a large number of round-corner portions neednot be stored in advance, it is necessary to perform processing ofgenerating a pattern using an n-degree function and matching processing,resulting in an increase in the processing time.

As described above, in the above-described conventional approaches,there are problems such that, for example, a large memory capacity isnecessary, processing requiring a long time for matching or the likemust be performed, accuracy in recognition is inferior, and a roundportion of rules is segmented as a character and is erroneouslyrecognized.

SUMMARY OF THE INVENTION

It is an object of the present invention to solve the above-describedproblems.

It is another object of the present invention to correctly extract acharacter within a frame for describing a character, even if a corner ofthe frame is round.

According to one aspect of the present invention, an image processingapparatus includes row-direction-rule extraction means for extractingrules in a row direction from a document image, column-direction-ruleextraction means for extracting rules in a column direction from thedocument image, image-region extraction means for extracting an imageregion based on a region surrounded by the row-direction rules extractedby the row-direction-rule extraction means and the column-directionrules extracted by the column-direction-rule extraction means, andremoval means for removing pixels constituting a part of rules from theimage region extracted by the image-region extraction means, byidentifying whether or not a desired pixel group contacting at least oneof the row-direction rules and the column-direction rules constitutespart of the rules, by performing labeling for the pixel group.

According to another aspect of the present invention, an imageprocessing apparatus includes an image scanner and a processor. Theprocessor is configured to (i) extract rules in a row direction from adocument image scanned by said image scanner, (ii) extract rules in acolumn direction from the document image, (iii) extract an image regionbased on a region surrounded by the extracted row-direction rules andthe extracted column-direction rules, and (iv) remove pixels from theextracted image region that constitute part of rules, by identifyingwhether a desired pixel group contacting at least one of therow-direction rules and column-direction rules constitutes part of therules, by performing labeling for the pixel group.

According to still another aspect of the present invention, an imageprocessing method includes a row-direction-rule extraction step ofextracting rules in a row direction from a document image, acolumn-direction-rule extraction step of extracting rules in a columndirection from the document image, an image-region extraction step ofextracting an image region based on a region surrounded by therow-direction rules extracted in the row-direction-rule extraction stepand the column-direction rules extracted in the column-direction-ruleextraction step, and a removal step of removing pixels constituting apart of rules from the image region extracted in the image-regionextraction step, by identifying whether or not a desired pixel groupcontacting at least one of the row-direction rules and thecolumn-direction rules constitutes part of the rules, by performinglabeling for the pixel group.

According to yet another aspect of the present invention, an imageprocessing program, capable of being executed by a computer, includesprogram codes for causing a computer to realize steps, the stepsincluding a row-direction-rule extraction step of extracting rules in arow direction from a document image, a column-direction-rule extractionstep of extracting rules in a column direction from the document image,an image-region extraction step of extracting an image region based on aregion surrounded by the row-direction rules extracted in therow-direction-rule extraction step and the column-direction rulesextracted in the column-direction-rule extraction step, and a removalstep of removing pixels constituting a part of rules from the imageregion extracted in the image-region extraction step, by identifyingwhether or not a desired pixel group contacting at least one of therow-direction rules and the column-direction rules constitutes part ofthe rules, by performing labeling for the pixel group.

According to yet a further aspect of the present invention, a storagemedium stores an image processing program, capable of being executed bya computer, including program codes for causing a computer to realizesteps, the steps including a row-direction-rule extraction step ofextracting rules in a row direction from a document image, acolumn-direction-rule extraction step of extracting rules in a columndirection from the document image, an image-region extraction step ofextracting an image region based on a region surrounded by therow-direction rules extracted in the row-direction-rule extraction stepand the column-direction rules extracted in the column-direction-ruleextraction step, and a removal step of removing pixels constituting apart of the rules from the image region extracted in the image-regionextraction step, by identifying whether or not a desired pixel groupcontacting at least one of the row-direction rules and thecolumn-direction rules constitutes part of the rules, by performinglabeling for the pixel group.

Other features and advantages of the patent invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention.

FIG. 1 is a block diagram illustrating an image processing apparatusaccording to a first embodiment of the present invention;

FIG. 2 is a flowchart illustrating a procedure for segmenting acharacter according to the first embodiment;

FIG. 3 is a flowchart illustrating a procedure for removing a cornerrule portion, according to the first embodiment;

FIG. 4 is a diagram illustrating a frame for describing characters,according to the first embodiment;

FIG. 5 is a diagram illustrating processing for extracting intra-rulerectangular regions, according to the first embodiment;

FIGS. 6A and 6B are diagrams illustrating an example in which a part ofrules is included in the region of a circumscribed rectangle of blackpixels, according to the first embodiment;

FIGS. 7A and 7B are diagrams illustrating a result of labeling anintra-rule region, according to the first embodiment;

FIG. 8 is a flowchart illustrating a procedure for removing a cornerrule portion, according to a second embodiment of the present invention;

FIG. 9 is a diagram illustrating a frame for describing a character anda described character in the second embodiment; and

FIG. 10 is a diagram illustrating a memory map of a storage mediumstoring various processing programs which can be read by a documentreading apparatus according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

(First Embodiment)

FIG. 1 is a block diagram illustrating the configuration of an imageprocessing apparatus for reading characters from a document image,according to a first embodiment of the present invention. In FIG. 1, aCPU (central processing unit) 101 controls the entirety of the apparatusin accordance with a control program stored in a ROM (read-only memory)102. The ROM 102 stores control programs and the like, which controlprocesses shown in flowcharts to be described later, to be executed bythe CPU 101. A RAM (random access memory) 103 stores document images andthe like, and operates as a working area for the CPU 101. An externalstorage device 104 comprises a magnetic disk or the like. There are alsoshown a display 105, a keyboard 106, a pointing device 107, such as amouse or the like, and an image scanner 108 for reading an image. Anetwork interface 109 communicates with an apparatus present at a remotelocation (not shown), and reads and writes programs, data and the like.In order to realize the first embodiment, the CPU 101 may execute acontrol program by reading it from the ROM 102, from the externalstorage device 104, or an external apparatus connected via a network.

Operation of the first embodiment, as executed by the image processingapparatus shown in FIG. 1, will now be described in detail withreference to FIGS. 1–7.

The first embodiment deals with a document in which a frame fordescribing a character is set for each character as shown in FIG. 4. Inorder to simplify description, the first embodiment deals with adocument image obtained by binary coding a read image.

FIG. 2 is a flowchart illustrating a procedure for segmenting adescribed character from a frame region within a document image.

In step S201, the positions of rules in the horizontal direction(hereinafter abbreviated as “horizontal rules”) are detected from animage read from a photoelectric transducer, such as the image scanner108, and subjected to binary coding (hereinafter termed a “binaryimage”), or from a binary image that has been read and stored in theexternal storage device 104 or the like.

As for the position of a horizontal rule, in order to represent thecoordinate values of the space of the horizontal rule in the verticaldirection, and the thickness of the rule, the horizontal rule isrepresented by the coordinate values of the upper and lower ends of thehorizontal rule in the vertical direction.

For example, when detecting the positions of a horizontal rule from aregion for describing characters as shown in FIG. 4, in a simpledetection procedure, a horizontal-direction histogram of black pixelswith respect to a vertical projection plane is obtained, and thepositions of rise and fall of the peak of a portion where the frequencyof peaks equals at least a predetermined threshold are made thepositions of the horizontal rule. In order to improve accuracy indetection, a region of processing may be determined while removinginclination, complementing a thin portion, and extracting acircumscribed rectangle of a frame for describing a character inadvance. In this case, for example, the threshold may be set to ½ of thewidth of the line of the circumscribed rectangle. For detection of thepositions of a horizontal rule, for example, a technique described inJapanese Patent Application Laid-Open (Kokai) No. 2000-380612 (2,000)filed by the assignee of the present application may be used.

In step S202, it is determined whether or not at least two horizontalrules have been extracted. If the result of the determination in stepS202 is negative, the process is terminated assuming that a frame fordescribing a character is absent.

If the result of the determination in step S202 is affirmative, theprocess proceeds to step S203, where the positions of vertical rulespresent between two horizontal rules, i.e., between the coordinate ofthe lower portion of the upper horizontal rule and the coordinate of theupper portion of the lower horizontal rule, are detected.

For detecting the positions of a vertical rule, a vertical-directionhistogram of black pixels with respect to a horizontal projection planeis obtained, and a portion where the frequency of peaks equals at leasta threshold are identified as the position of the vertical rule. Forexample, the threshold may be set to at least 80% of the distancebetween the horizontal rules. For detection of the position of avertical rule, for example, a technique described in Japanese PatentApplication Laid-Open (Kokai) No. 2000-380612 (2,000) may also be used.

In step S204, it is determined whether or not at least two verticalrules have been extracted. If the result of the determination in stepS204 is negative, the process is terminated assuming that a frame fordescribing a character is absent.

If the result of the determination in step S204 is affirmative, theprocess proceeds to step S205, where the inner portion of a rectangularregion surrounded by the extracted horizontal rules and vertical rulesis extracted as an intra-rule rectangular region. FIG. 5 illustrates aresult of extracting intra-rule rectangular regions from the documentshown in FIG. 4, using the above-described processing. At that time,since “1” is recognized as a vertical rule, two divided intra-rulerectangular regions 504 and 505 are present.

In step S206, a standard pitch of a frame for describing a character isobtained from the extracted intra-rule rectangular region. The standardpitch is determined, for example, from the distribution of the pitchesof respective intra-rule rectangular regions. That is, based on theassumption that the pitch of each frame for describing a characterwithin a range of processing is uniform, the distribution of the pitchof each intra-rule rectangular region is counted, and a pitch whosenumber is largest is adopted as the standard pitch.

When regions having a pitch smaller than the standard pitch aredetermined to constitute an erroneously divided region, then, in stepS207, these regions are connected in order to have the standard pitch.As a result, the regions 504 and 505 shown in FIG. 5 are connected, anda region which coincides with the intra-rule rectangular region isobtained.

In step S208, black pixels within an intra-rule rectangular region aredetected, and an image is extracted using a circumscribed rectanglesurrounding the black pixels within the intra-rule rectangular region.An image is to be extracted using one circumscribed rectangle for oneintra-rule rectangular region. In order to remove influence ofisolated-point noise, noise contacting a rule, and the like, well-knownnoise removal may be performed. In another approach, when an extractedimage is smaller than a predetermined size, the image may be assumed tobe not present by determining that the image is noise.

In step S209, it is determined whether or not an image is present withinthe intra-rule rectangular region. If the result of the determination instep S209 is negative, the process proceeds to step S213, where thecorresponding frame for describing a character is made a void.

If the result of the determination in step S209 is affirmative, theprocess proceeds to step S210, where it is determined whether or not apart of a rule is included within the extracted image. If the result ofnegative, the image is set as a character image in step S214. If theresult of the determination in step S210 is affirmative, that portion isremoved in step S211. After the removal, in step S212 it is determinedwhether an image remains and flow moves to step S213 or step S214 inaccordance with the result of the determination.

FIGS. 6A and 6B illustrate a case in which a part of one or more rulesis contained within a region. The case of an intra-rule rectangularregion 508 obtained in step S207 will now be described. First, arectangular image within the intra-rule rectangular region 508 issegmented. A circumscribed rectangle of black pixels within thesegmented rectangular region is represented by a broken-line portion 601shown in FIG. 6A. As a result, as shown in FIG. 6B, portions 602 and 603of rules remain in the image within the circumscribed rectangle 601. Ifcharacter recognition is performed for the image shown in FIG. 6B,erroneous recognition will be performed. That is, when round corners arepresent, if it is intended to extract an image within the frame based onthe intra-rule rectangular region, the round portions (each of which isa part of a rule) are included.

Removal when a part of a round rule is included within an extractedimage (step S211) will now be described in detail illustrating a case inwhich an upper right corner is round, with reference to the flowchartshown in FIG. 3. A description will be provided assuming that thecoordinates of the intra-rule rectangular region 508 obtained in stepS207, and the coordinates of the circumscribed rectangle 601 of theblack pixels obtained based on the intra-rule rectangular region 508 instep S208 are stored. It is assumed that the uppermost position at theleft of the document image is made an origin of the coordinates, thex-coordinate value increases from the left to the right, and they-coordinate value increases from top to bottom.

In step S301, for the intra-rule rectangular region 508, the ycoordinate of the upper end is represented by T, the x coordinate of theright end is represented by R, the height of the region is representedby H, and the width of the region is represented by W. In step S302, they coordinate of the upper end, and the x coordinate of the right end ofthe circumscribed rectangle 601 of the extracted black pixels arerepresented by t and r, respectively.

In step S303, it is determined whether or not the x coordinate R of theright end of the intra-rule rectangular region 508 equals the xcoordinate r of the right end of the circumscribed rectangle 601 of theblack pixels. If the result of the determination in step S303 isaffirmative, the process proceeds to step S304, where it is determinedwhether or not the upper end of the circumscribed rectangle 601 of theblack pixels is at a position higher than a position ⅙ of the heightbelow the upper end of the intra-rule rectangular region 508. If theresult of the determination in step S304 is affirmative, the processproceeds to step S307. If the result of the determination in step S304is negative, the process is terminated. The above-described value of ⅙,and respective thresholds to be described below, are only examples forexplaining the present invention. The present invention is not limitedto these values.

If the result of the determination in step S303 is negative, the processproceeds to step S305, where it is determined whether or not the ycoordinate T of the upper end of the intra-rule rectangular region 508equals the y coordinate t of the upper end of the circumscribedrectangle 601 of the black pixels. If the result of the determination instep S305 is negative, the process is terminated. If the result of thedetermination in step S305 is affirmative, the process proceeds to stepS306, where it is determined whether or not the right end of thecircumscribed rectangle 601 of the black pixels is to the right of aposition that is ⅙ of the width to the left of the right end of theintra-rule rectangular region 508. If the result of the determination instep S306 is affirmative, the process proceeds to step S307. If theresult of the determination in step S306 is negative, the process isterminated.

In step S307, black pixels contacting the upper end of the intra-rulerectangular region 508 within a right-half portion thereof are subjectedto labeling, and black pixels at the lowermost end of black pixelshaving the same label within an upper-half portion of the intra-rulerectangular region 508 (those at positions above ½ of the height) areextracted.

In step S308, black pixels contacting the right end of the intra-rulerectangular region 508 within an upper-half portion thereof aresubjected to labeling, and black pixels at the left end of black pixelshaving the same label within a right-half portion of the intra-rulerectangular region 508 (those to the right of ½ of the width) areextracted.

In step S309, a circumscribed rectangle of all labeled pixels isobtained. In the case shown in FIGS. 7A and 7B, FIG. 7A illustrates anoriginal image, and FIG. 7B illustrates an image after labeling.According to labeling processing, two blocks having label 1 and label 2are extracted. The obtained circumscribed rectangle is a rectangleincluding the two blocks.

In step S310, a straight line passing through the upper left corner andthe lower right corner of the obtained circumscribed rectangle isobtained. In step S311, the numbers of labeled pixels to the right andto the left of the straight line are counted. If the pixels constitute arule, most of the pixels are to be at the right of the straight line.Hence, it is possible to determine whether the pixels constitute a ruleby comparing the numbers of labeled pixels to the right and to the leftof the straight line.

In step S312, it is determined whether or not the number of pixels atthe right is smaller than four times the number of pixels at the left.If the result of the determination in step S312 is affirmative, theprocess ends. If the result of the determination in step S312 isnegative, the process proceeds to step S313, where labeled images areremoved from the circumscribed rectangle 601. In the case shown in FIG.7, since pixels are absent to the left of the straight line for bothlabel 1 and label 2, the pixels having these two labels are removed.

For each of lower right, upper left and lower left corners, byperforming the same processing by changing the direction and variables,it is possible to remove only rule portions even if an image isextracted inclusive of a part of rules.

Character recognition processing is performed for an image not includingrules that has been obtained in the above-described manner. Since a ruleportion is not included, a character described in a document can becorrectly recognized.

As described above, according to the first embodiment, since blackpixels contacting a border of an intra-rule region are subjected tolabeling, and pixels are removed by applying determination criteria foreach group of labeled pixels, a rule portion can be removed even ifrules are disconnected at a round portion.

According to the first embodiment, since the determination criteria areobtained from the coordinates of labeled pixels, a character image canbe extracted from a frame for describing a character so as not toinclude rules, even if the size of a round corner changes, or theroundness changes to an elliptical arc.

Since it is unnecessary to store the pattern of each corner, thecapacity of a memory can be economized. Furthermore, it is unnecessaryto perform matching processing with a plurality of corner patterns, andtherefore the processing time can be reduced.

(Second Embodiment)

A description will now be provided of a case in which a corner ruleportion is removed in step S211 when a part of a character is superposedon a corner, according to a second embodiment of the present inventionexecuted by the document reading apparatus shown in FIG. 1, withreference to the flowchart shown in FIG. 8, and the drawing in FIG. 9.

FIG. 9 illustrates a frame for describing a character that is to beprocessed in the second embodiment. The upper right corner is rounded,and a part of a character is described there in a state of exceeding theframe boundary.

A processing procedure when the upper right corner is round will now bedescribed.

Referring to FIG. 8, in step S801, for an intra-rule rectangular regionobtained in the same manner as in the first embodiment, the y coordinateof the upper end is represented by T, the x coordinate of the right endis represented by R, the height of the region is represented by H, andthe width of the region is represented by W. In step S802, the ycoordinate of the upper end and the x coordinate of the right end of acircumscribed rectangle of extracted black pixels are represented by tand r, respectively.

In step S803, it is determined whether or not the x coordinate R of theright end of the intra-rule rectangular region equals the x coordinate rof the right end of the circumscribed rectangle of the black pixels. Ifthe result of the determination in step S803 is affirmative, the processproceeds to step S804, where it is determined whether or not the upperend of the circumscribed rectangle of the black pixels is at a positionhigher than a position ⅙ of the height below the upper end of theintra-rule rectangular region. If the result of the determination instep S804 is affirmative, the process proceeds to step S807. If theresult of the determination in step S804 is negative, the process isterminated. The above-described value of ⅙, and respective thresholds tobe described below, are only examples for explaining the presentinvention. The present invention is not limited to these values.

If the result of the determination in step S803 is negative, the processproceeds to step S805, where it is determined whether or not the ycoordinate T of the upper end of the intra-rule rectangular regionequals the y coordinate t of the upper end of the circumscribedrectangle of the black pixels. If the result of the determination instep S805 is negative, the process is terminated. If the result of thedetermination in step S805 is affirmative, the process proceeds to stepS806, where it is determined whether or not the right end of thecircumscribed rectangle of the black pixels is to the right of aposition that is ⅙ of the width to the left of the right end of theintra-rule rectangular region. If the result of the determination instep S806 is affirmative, the process proceeds to step S807. If theresult of the determination in step S806 is negative, the process isterminated.

In step S807, the upper end of the intra-rule rectangular region withina right-half portion is scanned, and the position of a black pixel atthe left end position is obtained (901). In step S808, the right end ofthe intra-rule rectangular region within an upper-half portion isscanned, and the position of a black pixel at the lowermost position isobtained (902).

In step S809, only pixels at upper right portions of a straight line(indicated by broken line 904) connecting the obtained two positions aresubjected to labeling.

In step S810, a label including a pixel at the right end that contactsthe upper end of the intra-rule rectangular region from among labeledpixels is selected.

In step S811, a label including a pixel at the upper end that contactsthe right end of the intra-rule rectangular region from among labeledpixels is selected. If a thin portion and the like are absent in rules,the label selected in step S810 is usually the same as the labelselected in step S811.

When a plurality of labels have been selected in steps S810 and S811,then, in step S812, it is determined whether or not the labels areclose. If the result of the determination in step S812 is negative, theconcerned image is determined to be not the image of a rule at a cornerportion, and the process is terminated. Whether or not the labels areclose is determined by checking whether or not circumscribed rectanglesof labeled black-pixel images are superposed, or whether or not thedistance between respective circumscribed rectangles is within apredetermined number of pixels (for example, one pixel).

In step S813, the position of a pixel at the left end that contacts theupper end of the inter-rule region from the selected label is obtained(901).

In step S814, the position of a pixel at the lower end that contacts theright end of the inter-rule region from the selected labels is obtained(903).

If a described character or the like does not contact a rule at aposition other than a round corner, the position of the pixel obtainedin step S813 is the same as the position of the pixel obtained in stepS807, and the position of the pixel obtained in step S814 is the same asthe position of the pixel obtained in step S808. At this stage, thestarting position of a round corner included in the extracted image isobtained.

When the position obtained in step S813 is represented by (X,T), and theposition obtained in step S814 is represented by (R,Y), then in stepS815, it is determined whether or not the selected labeled image isdivided by a straight line (indicated by broken lines 905) connectingthe positions (X,T) and (R,Y). If a character crossing a rule is absent,the image is not divided. If the result of the determination in stepS815 is affirmative, the process proceeds to step S816, where thecontour of labeled pixels is tracked from the point of division 906 in adirection (R,T), and points 907 and 908 where the direction of thecontour greatly changes are searched. If the width of a line crossing arule corresponds to at least two pixels, two pixels are present on thecontour, and the two pixels are tracked. If the width corresponds to onepixel, the contour of the upper portion and the contour of the lowerportion of the pixel on the contour are tracked.

As a result, the two points 907 and 908 where the direction of thecontour changes are obtained. In step S817, the labeled image is dividedby a straight line connecting the two points, and an image present atthe upper right side is left as a labeled image.

The left labeled image is determined to be a part of a rule, and isremoved in step S818. Thus, it is possible to obtain an image in which around corner of rules is removed, and perform character recognitionprocessing for the obtained image.

As in the first embodiment, for each of lower right, upper left andlower left corners, by performing the same processing, rule portions areremoved. As a result, it is possible to extract a character image inwhich rule portions are removed.

As described above, according to the second embodiment, since labelingis performed by limiting the region of an image to be processed, and theposition of a round corner of a rule is specified, it is possible toremove only rule portions and leave a character within a frame, evenwhen a part of the character is threaded through the round portion.

(Other Embodiments)

The present invention may be applied to a system comprising a pluralityof apparatuses (such as a host computer, an interface apparatus, areader, a printer and the like), or to an apparatus comprising a singleunit (such as a copier, a facsimile apparatus or the like).

The objects of the present invention may, of course, also be achieved bysupplying a system or an apparatus with a storage medium (or a recordingmedium) storing program codes of software for realizing the functions ofthe above-described embodiments, and reading and executing the programcodes stored in the storage medium by means of a computer (or a CPU oran MPU (microprocessor unit)) of the system or the apparatus. In such acase the program codes themselves read from the storage medium realizethe functions of the above-described embodiments, so that the storagemedium storing the program codes constitutes the present invention. Thepresent invention may, of course, be applied not only to a case in whichthe functions of the above-described embodiments are realized byexecuting program codes read by a computer, but also to a case in whichan OS (operating system) or the like operating in a computer executes apart or the entirety of actual processing, and the functions of theabove-described embodiments are realized by the processing.

The present invention may, of course, be applied to a case in which,after writing program codes read from a storage medium into a memoryprovided in a function expanding card inserted into a computer or in afunction expanding unit connected to the computer, a CPU or the likeprovided in the function expanding card or the function expanding unitperforms a part or the entirety of actual processing, and the functionsof the above-described embodiments are realized by the processing.

Furthermore, program codes constituting the present invention may beread from an external apparatus connected via a network whenevernecessary and may be executed by a CPU.

When applying the present invention to the above-described storagemedium, program codes corresponding to the above-described flowchartsare stored in the storage medium. FIG. 10 illustrates an example of sucha storage medium. In FIG. 10, a memory map of a storage medium storingvarious data processing programs capable of being read by a documentreading apparatus according to the present invention is shown. Althoughnot illustrated, sets of data to be used for various programs are alsomanaged in a directory of the storage medium. In some cases, forexample, a program for installing various programs is stored.

As described above, according to the present invention, even in a framefor describing a character including rules having round corners, a partof rules is not extracted as a character, and a character within theframe can be correctly extracted. Since a described character can becorrectly extracted, the rate of character recognition is improved.

According to the present invention, it is automatically determinedwhether or not a part of rules is included, and if the result of thedetermination is affirmative, that part is removed. Accordingly, it isunnecessary to assign the shape of each frame in advance.

The individual components shown in outline or designated by blocks inthe drawings are all well known in the image processing apparatus,method and program arts and their specific construction or operation arenot critical to the operation or the best mode for carrying out theinvention.

While the present invention has been described with respect to what arepresently considered to be the preferred embodiments, it is to beunderstood that the invention is not limited to the disclosedembodiments. To the contrary, the present invention is intended to covervarious modifications and equivalent arrangements included within thespirit and scope of the appended claims. The scope of the followingclaims is to be accorded the broadest interpretation so as to encompassall such modifications and equivalent structures and functions.

1. An image processing apparatus comprising: row-direction-ruleextraction means for extracting rules in a row direction from a documentimage; column-direction-rule extraction means for extracting rules in acolumn direction from the document image; image-region extraction meansfor extracting an image region based on a region surrounded by therow-direction rules extracted by said row-direction-rule extractionmeans and the column-direction rules extracted by saidcolumn-direction-rule extraction means; and removal means for removingpixels constituting a part of rules from the image region extracted bysaid image-region extraction means, by identifying whether or not adesired pixel group contacting at least one of the row-direction rulesand the column-direction rules constitutes part of the rules, byperforming labeling for the pixel group.
 2. An image processingapparatus according to claim 1, wherein said removal means removes thepixels constituting part of the rules from the image region by setting aborder based on the labeled pixel group, measuring respective numbers ofpixels on opposite sides of the border in the labeled pixel group, andidentifying whether or not the labeled pixel group constitutes part ofthe rules based on a result of the measurement.
 3. An image processingapparatus according to claim 1, wherein said removal means removes thepixels constituting part of the rules from the image region by setting aborder based on the labeled pixel group, identifying a position wherethe border and the labeled pixel group intersect, tracking a contourline from the position of intersection, and identifying pixelsconstituting part of the rules from the labeled pixel group based on aresult of tracking the contour line.
 4. An image processing apparatusaccording to claim 1, wherein the labeling is performed only for pixelswithin a predetermined region of the image region.
 5. An imageprocessing apparatus according to claim 1, further comprising characterrecognition means for performing character recognition of an imagewithin the image region from which the pixels constituting part of therules have been removed by said removal means.
 6. An image processingmethod comprising: a row-direction-rule extraction step of extractingrules in a row direction from a document image; a column-direction-ruleextraction step of extracting rules in a column direction from thedocument image; an image-region extraction step of extracting an imageregion based on a region surrounded by the row-direction rules extractedin said row-direction-rule extraction step and the column-directionrules extracted in said column-direction-rule extraction step; and aremoval step of removing pixels constituting a part of rules from theimage region extracted in said image-region extraction step, byidentifying whether or not a desired pixel group contacting at least oneof the row-direction rules and the column-direction rules constitutespart of the rules, by performing labeling for the pixel group.
 7. Animage processing method according to claim 6, wherein in said removalstep, the pixels constituting part of the rules are removed from theimage region by setting a border based on the labeled pixel group,measuring respective numbers of pixels on opposite sides of the borderin the labeled pixel group, and identifying whether or not the labeledpixel group constitutes part of the rules based on a result of themeasurement.
 8. An image processing method according to claim 6, whereinin said removal step, the pixels constituting part of the rules areremoved from the image region by setting a border based on the labeledpixel group, identifying a position where the border and the labeledpixel group intersect, tracking a contour line from the position ofintersection, and identifying pixels constituting part of the rules fromthe labeled pixel group based on a result of tracking the contour line.9. An image processing method according to claim 6, wherein the labelingis performed only for pixels within a predetermined region of the imageregion.
 10. An image processing method according to claim 6, furthercomprising a character recognition step of performing characterrecognition of an image within the image region from which the pixelsconstituting the part of the rules have been removed in said removalstep.
 11. A computer-readable storage medium storing an image processingprogram, capable of being executed by a computer, including programcodes for causing a computer to realize steps, said steps comprising: arow-direction-rule extraction step of extracting rules in a rowdirection from a document image; a column-direction-rule extraction stepof extracting rules in a column direction from the document image; animage-region extraction step of extracting an image region based on aregion surrounded by the row-direction rules extracted in saidrow-direction-rule extraction step and the column-direction rulesextracted in said column-direction-rule extraction step; and a removalstep of removing pixels constituting a part of rules from the imageregion extracted in said image-region extraction step, by identifyingwhether or not a desired pixel group contacting at least one of therow-direction rules and the column-direction rules constitutes part ofthe rules, by performing labeling for the pixel group.
 12. Acomputer-readable storage medium according to claim 11, wherein in saidremoval step, the pixels constituting part of the rules are removed fromthe image region by setting a border based on the labeled pixel group,measuring respective numbers of pixels on opposite sides of the borderin the labeled pixel group, and identifying whether or not the labeledpixel group constitutes part of the rules based on a result of themeasurement.
 13. A computer-readable storage medium according to claim11, wherein in said removal step, the pixels constituting part of therules are removed from the image region by setting a border based on thelabeled pixel group, identifying a position where the border and thelabeled image group intersect, tracking a contour line from the positionof intersection, and identifying pixels constituting part of the rulesfrom the labeled pixel group based on a result of tracking the contourline.
 14. A computer-readable storage medium according to claim 11,wherein the labeling is performed only for pixels within a predeterminedregion of the image region.
 15. A computer-readable storage mediumaccording to claim 11, further comprising a character recognition stepof performing character recognition of an image within the image regionfrom which the pixels constituting part of the rules have been removedin said removal step.
 16. An image processing apparatus comprising: animage scanner; and a processor, said processor being configured to (i)extract rules in a row direction from a document image scanned by saidimage scanner, (ii) extract rules in a column direction from thedocument image, (iii) extract an image region based on a regionsurrounded by the extracted row-direction rules and the extractedcolumn-direction rules, and (iv) remove pixels from the extracted imageregion that constitute part of rules, by identifying whether a desiredpixel group contacting at least one of the row-direction rules andcolumn-direction rules constitutes part of the rules, by performinglabeling for the pixel group.
 17. An image processing apparatusaccording to claim 16, wherein said processor is configured to removethe pixels constituting part of the rules by setting a border based onthe labeled pixel group, measuring respective numbers of pixels onopposite sides of the border in the labeled pixel group, and identifyingwhether the labeled pixel group constitutes part of the rules based on aresult of the measurement.
 18. An image processing apparatus accordingto claim 16, wherein said processor is configured to remove the pixelsconstituting part of the rules by setting a border based on the labeledpixel group, identifying a position where the border and the labeledpixel group intersect, tracking a contour line from the position ofintersection, and identifying pixels constituting part of the rules fromthe labeled pixel group based on a result of tracking the contour line.19. An image processing apparatus according to claim 16, wherein theprocessor is configured to perform labeling only for pixels within apredetermined region of the image region.
 20. An apparatus according toclaim 16, wherein said processor is further configured to performcharacter recognition of an image within the image region from which thepixels constituting part of the rules have been removed.